A generalization of the entropy power inequality with applications

We prove the following generalization of the Entropy Power Inequality: h(Ax) h(A~ x) where h() denotes (joint-) diierential-entropy, x = x 1 : : : x n is a random vector with independent components, ~ x = ~ x 1 : : : ~ x n is a Gaussian vector with independent components such that h(~ x i) = h(x i), i = 1 : : : n, and A is any matrix. This generalization of the entropy-power inequality is applied to show that a non-Gaussian vector with independent components becomes \closer" to Gaussianity after a linear transformation, where the distance to Gaussianity is measured by the information divergence. Another application is a lower bound, greater than zero, for the mutual-information between non overlapping spectral components of a non-Gaussian white process. Finally, we describe a dual generalization of the Fisher Information Inequality. 0 1 The Generalization of the Entropy Power Inequality Consider the (joint-) diierential-entropy h(Ax), of a linear transformation y = Ax, where x = x 1 : : : x n is a vector and h(y) = Ef? log f(y)g (1) where we assume that y has a density f(). Throughout the manuscript log x = log 2 x and the entropy is measured in bits. Assume that dim A = m 0 n and RankA = m. In some cases, this entropy is easily calculated or bounded: 1. A is an invertible matrix (i.e., m 0 = m = n). In this case the lineartransformation just scales and shuues x, thus the entropy is only shifted, h(Ax) = h(x) + log jAj (2) where j j denotes (absolute value of) determinant. 2. A does not have a full row-rank (i.e., m 0 > m). In this case there is a deterministic relation between the components of y and thus h(Ax) = ?1 : (3) 3. x = x is a Gaussian vector. The linear transformation A preserves the normality and so h(Ax) = m 2 log(2ejAR x A t j 1 m) (4) where R x is the covariance matrix of x and AR x A t is the covariance matrix of y = Ax. Since for a given covariance, the Gaussian distribution maximizes the entropy, the expression in (4) upper bounds the entropy of y = Ax in the general case, i.e., h(Ax) h(Ax) (5) where x is now a Gaussian vector with the same covariance …